1. Field of the Invention
The invention relates generally to computer assisted, automated testing systems and procedures and more specifically relates to improved methods and structures for automated testing of embedded systems by providing a standardized user interface and API for communications between the embedded system under test, the test operator and an associated automated test host system.
2. Discussion of Related Art
It is critical in the electronic arts, and other technological arts, to verify design and operation of products and systems prior to volume manufacturing and distribution of such products. Further, as products evolve with enhancements and modifications, it is similarly critical to verify design and operation of the products so modified. In one form of testing, the system or product under test is viewed as a “black box” such that external information or stimuli are provided as inputs to the black box (system under test) and resulting actions or output values are confirmed as proper actions or outputs of the black box in response to appropriately applied stimuli. The internal structure and operation inside the “black box” are not relevant to such testing.
Such “black box” of testing is adequate to verify operation of the system in response to numerous expected environmental circumstances and stimuli. However, such black box testing often is insufficient to apply stressful stimuli outside normal expected ranges of operation.
In view of the weaknesses and limitations of black box testing, other forms of testing (sometimes referred to as “white box” testing) are useful to more effectively verify operation of the system under test in response to a wider range of stimuli including those associated with conditions generated internal to the system under test. Exemplary of such testing approaches is the automated testing of embedded systems. For example, it is common that intelligent I/O controller devices inserted within a host system often possess substantial computational and processing power as integrated circuit chips designed within the intelligent I/O controller. For example, an intelligent I/O controller may include one or more general-purpose processing devices (i.e., CPUs), one or more special-purpose processors (i.e., I/O coprocessors), and substantial amounts of memory for storing code to operate the various processors, variables associated with that operation and buffer memory for buffering user data moving into, or out of, the intelligent I/O controller. To effectively test operation of all of these components often requires that special test programs be designed and loaded into the program memory associated with the intelligent I/O controller so as to enable the various processors within such an intelligent I/O controller to perform tests applying internally generated stimuli.
As presently practiced in the art, no standards exist for development, and operation of such embedded test sequences. Rather, individual test engineers often design unique test cases by producing appropriate test code instructions and developing unique host interfaces for communicating parameters of the test and test results between the host system in use by the test operator and the embedded systems under test. Creation of such a unique interface between systems under test and host systems with which a test operator interacts can create confusion in the style of interaction and hence errors in test procedures. Further, such unique design efforts generate duplicative work by multiple engineers each creating unique embedded test interfaces rather than sharing a common, standard interface.
It is evident from the above discussion that a need exists for improved methods and structures for generating, configuring and operating test sequences especially on embedded systems under test.